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ABSTRACT 



This update provides information about the Nebraska 
Statewide Writing Assessment follow-up, the federal reporting of assessment 
information, and preparing the school District Assessment Portfolio. The 
information is intended to assist local educators in the reporting of local 
assessment information in June 2001. The update on the writing assessment 
discusses the scores that meet Nebraska state writing standards and outlines 
the reports school districts will receive when the scoring process is 
complete. The section on federal reporting requirements discusses the 
reporting of student performance on reading, speaking, and listening 
standards due to the state in June 2001 and the reports due the federal 
government for limited English speakers and students receiving Title I 
services. The third section of the report concerns preparing the school 
district for the Assessment Portfolio each district must submit to describe 
how it has met the quality criteria for assessment defined in May 2000 during 
the 2000-2001 school year. (SLD) 
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Purpose of STARS Update #5 

STARS UPDATE #5 

This Update provides information about: 

I. Statewide Writing Assessment Follow-up 

II. Federal Reporting of Assessment Information 

III. Preparing the School District Assessment Portfolio 

This information is intended to assist local educators in the reporting of local assessment 
information in June, 2001. 

PREVIOUS STARS MATERIALS 

This Update continues the series of materials provided by the Nebraska Department of 
Education to assist in the implementation of the School-based Teacher-led Assessment 
and Reporting System (STARS). The series includes: 

• Nebraska School-based Teacher-led Assessment and Reporting Systems (STARS) 
Guide of October 1999 

• STARS Toolkit distributed in December 1999 (no longer in print) 

• STARS Update #1 in May 2000 (Moving Forward With Assessment) 

• STARS Update #2 in August 2000 (A STARS Summary) 

• STARS Update #3 in September 2000 (Writing Assessment) 

• STARS Update #4 in December 2000 (Writing Assessment, Reporting, Model 

Assessments) 

Schools are encouraged to maintain the complete set of the STARS Guide and Updates 
for reference in local planning. 
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This section provides information regarding 
the reporting of results in the statewide 
writing assessment. 
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I. Statewide W riting Assessment F ollow-up 



The following section provides a summary of information regarding the reporting of results of 
the Statewide Writing Assessment conducted in February, 2001. 

Meeting State Writing Standards 

Students will have met state writing standards if their scores are at or above the passing “cut 
score”. All reports sent back to districts from the regional scoring sites will identify that score 
and the number of students who have met it. 

Students who meet the cut score will have met the Nebraska writing standards with the following 
exceptions. Writing standards 4.2.5 and 8.2.5 (outlining and note taking) need to be measured in 
a locally developed assessment. 



Regional Scoring Report Information 



School districts participating in the regional scoring will receive three reports of their scoring 
results. The scoring results will include: 



District Summary Report by Grade Level 

Score Distribution Report by Building and Grade Level 

Building Grade Level Score Distribution Report by Student 



A sample of the District Summary Report by Grade Level is included in this Update. 
In considering the information included in the reports, educators are reminded that: 



• Each student paper was read and scored twice 



• The two scores were added together to determine the student’s final score. 
Therefore, the scores statewide will range between 2 and 8. 



Summary of Regional Scoring Report 

District Summary Report by Grade Level includes: 



Summary of score information for each grade assessed 

Number of students in each grade absent or taking an alternate assessment 

Local average score for each grade assessed 

Numbers and percentages of students scoring within each score point range 
Statewide cut score for each grade level assessed 
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• Percentage of local students in each grade scoring at or above the statewide cut 
score 

• Percentage of local students in each grade not scoring at or above the statewide 
cut score 

Score Distribution Report by Building and Grade Level includes: 

• Building and grade level average score 

• Number of students absent or taking an alternate assessment 

• Numbers and percentages of students scoring within each score point range 

• Statewide cut score for the grade level assessed 

• Percentage of local students scoring at or above the statewide cut score 

• Percentage of local students not scoring at or above the statewide cut score 

Building Grade Level Score Distribution Report by Student includes: 

• Total number of students by building and grade 

• Building and grade level average score 

• Statewide cut score for the grade level 

• Listing of students assessed by name and student identification number 

• Individual student scores 

• Students receiving accommodations 

• Students taking an alternate assessment 

• Additional score information 

Local Scoring Information 

In this pilot year of the Statewide Writing Assessment, school districts were given the option to 
conduct local scoring. Districts participating in local scoring: 

• Will receive materials and information to assist their scoring process. 

• May want to use information included in the regional scoring report format to 
assist them in creating their own local reports. 



Out of State Scoring 

A random sample of student writing assessments will be scored by reviewers outside Nebraska. 
The results of this scoring will provide an aggregate to be included in the State Report Card to be 
distributed in November, 2001. Individual school district’s results will not be included in the 
school district profiles to be published in the fall of 2001 . 
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Nebraska Statewide Writing Assessment 2001 

District Summary Report by Grade Level 
DEMOVILLE PUBLIC SCHOOL 



• 

Grade 


Total 

Students 


Alt. 

Assess./ 

Absent 








Scores 










Avg. 

Score 


No 

Score 


2-2.99 3-3.99 


4-4.99 


5-5.99 


6-6.99 


7-7.99 


8 


Grade 04 Students: 


15 


1 


1 


2 


3 


1 


2 


3 


2 


0 


A A A 


District Percentage: 




1 % 


7% 


13% 


20% 


7% 


13% 


20% 


13% 


0% 


4.40 


Grade 08 Students: 


15 


1 


1 


1 


3 


2 


2 


1 


4 


0 


A Q£L 


District Percentage: 




7% 


1 % 


1 % 


20% 


13% 


13% 


7% 


27% 


0% 


4.00 


Grade 11 Students: 


15 


0 


1 


1 


3 


2 


2 


2 


4 


0 


a on 


District Percentage: 




0% 


7% 


7% 


20% 


13% 


13% 


13% 


27% 


0% 


4.o7 


Students District-wide: 


45 


2 


3 


4 


9 


5 


6 


6 


10 


0 


4.71 


District Percentage: 




4% 


7% 


9 % 


20% 


11% 


13% 


13% 


22% 


0% 





Grade 04 Cut Score: Not Yet Available 

Students who score at or above the cut score have met all the writing standards except 4.2.5. 
Percentage of students who scored at or above the cut score: 64% 

Percentage of students who did not score at or above the cut score: 36% 




Gh^de 08 Cut Score: Not Yet Available 

^^»nts who score at or above the cut score have met all the writing standards except 8.2.5. 
Percentage of students who scored at or above the cut score: 57% 

Percentage of students who did not score at or above the cut score: 43% 

% 

Grade 11 Cut Score: Not Yet Available ^ 

Students who score at or above the cut score have met all the writing standards. 

Percentage of students who scored at or above the cut score: 67% 

Percentage of students who did not score at or above the cut score: 33% 



Graph Key: 

□ Grade 04 Percentages 

□ Grade 08 Percentages 

■ Grade 1 1 Percentages 

■ District Percentages 



✓ 
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This section provides information to assist 
school districts in preparing reports needed 
to meet federal reporting requirements. 
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II. Federal Assessment Reporting Requirements 



j 

The reporting of student performance on reading/speaking/listening standards is due 
June 30, 2001 to the Nebraska Department of Education. Schools should follow the 
reporting procedure as outlined in STARS Update #4. In addition, some federal 
programs have special reporting requirements. These are described below. 

Special Education Students 

General Information: 



• All special education students, with the exception of the small percentage of 
students taking an alternate assessment, will be included in the reports due on 
June 30, 2001 . In addition, there will be two added reports specifically for 
reporting special education students. These are described below. 



• Performance of all special education students will be reported to the State 
Department of Education, but performance of classes/grade levels with fewer 
than 1 0 students will not be published in the State of the Schools Report. 



Any district whose local standards and assessment plan have been approved as 
“equal to or more rigorous than the state’s” may report special education 
student progress on standards in grade levels other than 4, 8, and 11. 



Specialised Report One : Special Education Report 

• The first specialized report will include the reporting of only special education 
students in grades 4, 8, and 1 1 on reading/speaking/listening standards. 

• The only special education students not included in the Special Education 
Report on standards will be the small percentage of students taking an 
alternate assessment. 

• This report can be accessed electronically by clicking on the Special 
Education Report, (submit data! 

• Any district whose local standards and assessment plan have been approved as 
“equal to or more rigorous than” the state’s may report special education 
student progress in grade levels other than 4, 8, and 11. 
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Specialised Report Two : Alternate Assessment Report 



• Students participating in an alternate assessment (typically not more than 1% 
of the student population) will have assessment results included in the 
“Alternate Assessment Report.” 

• This report can be accessed electronically by clicking on: Alternate 
Assessment Report: (submit data) 



English Language Learners (ELL) 

General Information: 



• English Language Learners (ELL) refers to the same population of students 
termed limited English proficient (LEP) in federal legislation. 

• All English Language Learners will be included in the report due 

June 30, 2001 . In addition to including ELL students in the Total Student 
Report for grades 4, 8, and 1 1 on reading, speaking, and listening standards, 
there will be one added report for all English Language Learners. 

• Performance of all English Language Learners will be reported but 
performance of classes/grade levels with fewer than 1 0 students will not be 
published in the State of the School Report. 

• Any district whose local standards and assessment plan have been approved as 
“equal to or more rigorous than” the state’s may report ELL student progress 
on standards in grade levels other than 4, 8, and 1 1 . 

Specialised^ Report : English Language Learner Report 

• In addition to being included in the Total Student Report, English Language 
Learner students in grades 4, 8, and 1 1 will have a separate reporting form 
marked “English Language Learner Report.” Please note: English 
Language Learners who are given an alternate assessment will be 
included in both the Total Student Report and the ELL Report. 

• This report can be accessed electronically by clicking on the English 
Language Learner Report, (submit data) 

• Any district whose local standards and assessment plan have been approved as 
“equal to or more rigorous than” the state’s may report ELL student progress 
in grade levels other than 4, 8, and 1 1 . 
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Title I Programs 



• Students receiving Title I services are regular education students and are 
included in Total Student Report. 

• Title I reporting requirements will be provided by the Department of 
Education Title I Office to all schools having Title I programs. 



Career and Technical Education 

• Career and Technical Education students are regular education students and 
are included in the Total Student Report. 

• Career and Technical Education reporting requirements will be provided by 
the Department of Education Career and Technical Education Office to all 
schools having Career and Technical Education programs. 




For Further Information 

If you need additional information with regard to reporting on federal programs, 
please feel free to contact the following individuals at the Nebraska Department 
of Education: 



Ron Dughman 


Special Education 
rdughman(®,nde. state. ne. us 


402/471-6695 


Nancy Rowch 


English Language Learners 
nrowch(2),nde.state.ne.us 


402/471-2477 


Marilyn Peterson 


Title I 

mpeterso(®,nde. state, ne. us 


402/471-3504 


Steve Equall 
Michelle Kosmicki 


Career & Technical Education 
seauall(a),nde.state.ne.us 
mkosmicki(®,nde. state. ne. us 


402/471-2494 

402/471-4823 


Pat Roschewski 


Statewide Assessment 
proschew(@,nde. state. ne. us 


402/471-2495 
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This section provides information to assist 
school districts in preparing materials to be 
provided in the District Assessment 
Portfolio due to the State Department of 
Education on June 30, 2001 . 
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Guide for Preparing the 
District Assessment Portfolio 

Due to the Nebraska Department of Education June 30, 2001 



How Should This Guide Be Used? 

• This guide is intended to help districts continue to plan for submission of a 
District Assessment Portfolio due June 30, 2001 to the Nebraska Department of 
Education. 

• This guide is not the actual or official form to be submitted in June with the 
District Assessment Portfolios. Official forms will be mailed to each district in 
May, 2001. 

• The portfolios should describe how the district has met the Six Quality Criteria 
for assessment (provided to Nebraska districts in STARS Update #1 , May, 2000) 
during the 2000-01 school year. Therefore this guide provides recommendations 
for developing the Quality Criteria Description and organizing the portfolio. 



What is the District Assessment Portfolio? 



The District Assessment Portfolio will contain a collection of local assessment 
materials to be used for external review and rating of the local assessments. This 
review will guide Nebraska schools in improving local assessment instruments, 
procedures, and practices, and will partially fulfill the requirements of State Law 
79-760. 



• The District Assessment Portfolio should be prepared in two parts: (a) a 
collection of assessments used by a district to measure student progress on 
reading, speaking, and listening standards in grades 4, 8, and 11, and ( b) a 
description of how the district assessments meet the Six Quality Assessment 
Criteria. 

• If a district’s local standards have been approved as “equal to or more rigorous 
than” the state standards, the assessed grade levels may be different from 4, 8, 
and 11. 

• Some districts may not have had an opportunity to fully accomplish all of the 
Quality Criteria. In those instances, the district should provide as much 
information as is available. It is expected that assessment quality and procedures 
will improve as Nebraska educators gain expertise in assessment procedures. The 
review and rating should help all districts strengthen assessment knowledge and 
skill. 
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The District Assessment Inventory - Procedures for Determining the 
Assessments To Be Included in the District Portfolio 



In September 2000, school districts provided, in their local assessment plan, a chart of 
assessments to be used in 2000-01 to assess student progress in reading, speaking, and 
listening standards. The District Assessment Inventory, which has been mailed to 
Nebraska school districts, will update the September 2000 chart and should indicate the 
name or title of the local assessment actually used during the school year. 

In March, 2001 

• Each district will receive a “District Assessment Inventory” form for assessments 
used for assessing reading, listening, and speaking standards for grades 4, 8, 

and 11. 

• The District Assessment Inventory should be an update of the assessment plan 
submitted in September 2000. The inventory chart should be completed in order 
to include the names of all of the assessments used to measure the standards. 
Locally developed assessments should be identified by a locally determined name 
or title. 

• Districts whose standards and assessment plan have been approved as “equal to or 
more rigorous than” the state standards may complete a District Assessment 
Inventory on grades other than 4, 8, and 11. 

By April 17, 200f 

• The District Assessment Inventory is to be returned to the Nebraska Department 
of Education. 

• This inventory will be used to identify the assessments that are to be included in 
the District Assessment Portfolio. 

In Early May 

• Districts will receive a notification from the Nebraska Department of Education in 
early May! This notification will identify the assessments that are to be included 
in the District Assessment Portfolio. Districts may also include any district 
assessments that they consider to be exemplary. 

Assessments Developed by a Consortium 

• School districts that have developed local assessments as part of a consortium 
may submit just one District Assessment Inventory and one District Assessment 
Portfolio. The name of each school in the consortium must be attached and 

. identified by name and county district number. 
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What Should Be Included in the District Assessment Portfolio? 

• The contents of the, assessment portfolio should include: 

1 ) A cover sheet on an official form provided by Nebraska Department of 
Education. 

2) The Quality Criteria documentation requested by the Department of 
Education. 

3) The specific sample of local assessments requested and any additional 
local assessments that demonstrate high quality. 

• The contents should be assembled in three-ring binders with sections separated and 
divided by tabs. 

• The portfolio is to be submitted by June 30, 2001 . 



Each portfolio will be reviewed by an independent evaluation review team. Each school 
district will receive written feedback with comments and recommendations from the 
review team. 



How Will the Portfolio Materials Be Used? 





P.O. BOX 94987 
LINCOLN, NE 68509-4987. 
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DISTRICT ASSESSMENT PORTFOLIO 

for Reading, Speaking, and Listening 

June, 2001 



J 



District Name 



Consortium Name (if applicable) 

Schools who have participated in a consortium for the development of assessments may complete one Portfolio for the 
group. Consortiums need to identify all districts included by name and county district number. Attach a sheet if 
needed. 



County District Number . 



Address_ 






®f§ 

±4 




mm, 

MdneNumbe: 



*1 H#’ Jit 




mm 



Superintendent: 

Assessment Coordinator/Contact Person: 
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Assessment Quality Criteria 



Based upon a document prepared by the Buros Center for Testing, University of Nebraska-Lincoln 
for the Nebraska Department of Education 

This document is produced under a copyright from the Buros Center for Testing. It may be distributed to 
school districts in Nebraska. Other uses are prohibited. 

Quality Criterion One: The assessments reflect the state or local standards. 

A. Check the appropriate boxes. In order to have met this criterion, you will need to have 
completed item 1 or 2, and item 3. Check and describe all that apply. Add pages as 
needed. 

□ Yes . . . □ No. . . 1 . A panel of qualified teachers or other educators from the district has 

judged the assessments to be: (a) matched to the standard, and, (b) 
adequate to cover the standard. 

H ■ Your description may include: 

• numbers of teachers/educators/included in the panel 

• span of grade levels represented on the panel 

• content areas represented on the panel 

• number of years of experience represented on the panel 

• the process used by this panel 



□ Yes . . . □ No. . . 2. An outside group has judged the assessments to be: (a) matched to 

the standard, and, (b) adequate to cover the standard. 

• identify the group and numbers involved 

• describe the process used by the group 



and 



□ Yes ... □ No. . . 3. The assessment includes enough items or sufficient performance 

demonstrations for adequate coverage to determine the standard has been 
adequately covered. Therefore: 

• indicate the number of assessment items or demonstrations per 
standard 

• indicate how you know there is adequate coverage of the standard in 
the assessment 

• describe the scoring guidelines for subjectively scored tasks 



□ Yes ... □No./. 



4. Other: 



Quality Criterion Two: Students have an opportunity to learn the content. 



A. Check only the appropriate boxes that apply to your process. You do not need to do them 
all You must check and complete at least one and may check others. Add pages as 

needed. 



□ Yes. ..□No... 1. Representative panels of teachers from the grade and content area have 

examined the curriculum to determine that student have the opportunity to 
learn the content prior to test administration. Your description may 
include: 

• numbers of teachers/educators/included in the panel 

• the percentage of total teachers available who served on the panel 

• span of grade levels represented on the panel 

• content areas represented on the panel 

• number of years of experience represented on the panel 

• the process used by this panel 



□ Yes ... □ No. . . 2. A survey was conducted of grade and/or content appropriate teachers to 

determine where the test content is addressed in lesson plans. If this was 
done, you may include: 

• how many teachers were surveyed, what grade levels were 
represented 

• the response rate of the survey 

• a copy of the survey 



□ Yes ... □ No... 



□ Yes ... □ No... 



3. Panels of teachers or administrators have collected and examined 
samples of classroom assessments to determine that test content has been 
taught. Your description may include: 

• general qualifications of the panel or administrative team 

• the number/type of sample assessments collected 

• the percentages of the total samples that were collected 

4. Panels of teachers or administrators have examined or collected lesson 
plans to determine that test content has been taught. Your description may 
include: 

• general qualifications of the panel or administrative team 

• the number/type of sample lesson plans collected 

• the percentages of the total number of sample plans that were collected 

• any change you have made in your lesson planning process 



□ Yes ... □ No. . . 5. Peer teachers/administrators have conducted classroom observations to 

determine that test content has been taught. Your description may include: 

• any observation process used to determine that content is 
being taught 

• opportunities that staff have to receive feedback about 
content being taught 



□ Yes ... □ No... 6. Other: 



Quality Criterion Three: The assessments are free from bias. 




A. 



Check only the appropriate boxes that apply to your process. You do not need to do them 
all You must check and complete at least one and may check others. Add pages as 
needed. 



□ Yes ... □ No. . . 1 . The assessment writers have participated in an orientation regarding 

test bias. Your description may include: 

• the bias orientation that was conducted. 

• the name and qualifications of the individuals who conducted the 
orientation 



□ Yes ... □ No. . . 2. A panel of educators or others representing various socio-economic, 

and ethnic groups has reviewed drafts of the assessments. Your 
description may include: 

• a description of the panel of educators - what socio-economic, racial 
ethnic groups reviewed the assessments/number on the panel. 

• A description of the general qualifications of the panel members. 



□ Yes ... □No... 



3. A statistical analysis of the tests has been conducted to verify that 
items on the test do not demonstrate statistical bias. Your description 
may include: 

• an identification of the statistical analysis and indicate by whom it was 
done. 

• what the analysis showed. 

• Identification of who conducted the analysis. 



□ Yes ... □No... 4. Other: 
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Quality Criterion Four: The level is appropriate for students. 

A. Check only the appropriate boxes that apply to your process. You do not need to do them 
all You must check and complete at least one box and may check others. Add pages as 

needed. 



□ Yes ... □ No. . . 1 . A panel (s) of grade and/or content level teachers has examined and 

evaluated the assessments from a developmental cognitive perspective. 
Y our description may include: 

• numbers of teachers/educators/included in the panel 

• the percentage of total teachers available who served on the panel 

• span of grade levels represented on the panel 

• content areas represented on the panel 

• number of years of experience represented on the panel 

• the process used by this panel 



□ Yes ... □ No. . . 2. Outside specialists in reading, cognitive psychology, or other similar 

areas have examined and evaluated the assessment instruments from a 
developmental perspective. Your description may: 

• identify the specialists/ and number involved 

• include the general qualifications of the specialists 

• include the process used by the specialists 



□ Yes ... □ No. . . 3. Readability analyses have been conducted on the assessments. Your 

description may: 

• identify who conducted the analyses and their qualifications 

• identify the readability process used. 

• include the results of the analysis 



□ Yes ... □No... 4. Other: 



Quality Criteria Five: There is consistency in scoring. 




A. 



Check only the appropriate boxes that apply to your process. You do not need to do them 
alL You must check and complete at least one and may check others. Add pages as 
needed. 



□ Yes . . . □ No. . . 1 . Reliability has been estimated by (a) retest, (b) alternate forms of the 

assessment, or (c) an internal consistency reliability process. Your 
description may include: 

• the retest/altemative form in terms of number, type, format, 
administrative interval, etc. 

• the internal consistency reliability process used/who 
conducted it/how 

• the results of the reliability computation 



□ Yes . . . □ No. . . 2. A panel has screened and pre-tested the clarity and adequacy of the 

scoring rubric. Y our description may include: 

• numbers of teachers/educators/included in the panel 

• span of grade levels represented on the panel 

• content areas represented on the panel 

• number of years of experience represented on the panel 
the process used by this panel 



□ Yes . . . □ No . . . 3 . The scorers have been trained through the use of “marker” papers/ 

performances pre-scored by qualified trainers. Your description may: 

• Describe the trainers/number/qualifications 

• Describe the process used for the training 



□ Yes . . . □ No. . . 4. Sample papers/performances have been double scored to estimate the 

percent of scorer agreement. Your description may: 

• Describe the number of papers double scored. 

• Describe the process used for the double scoring 

• Identify the percentage of scorer agreement (both exact and adjacent) 



□ Yes ... □No... 5. Other: 
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Quality Criterion Six: Mastery levels are appropriate. 

A. Check only the appropriate boxes that apply to your process. 

□ Yes ... □ No. . . 1 . A panel of educators has examined the assessment tasks or items and 

determined the level of performance required to achieve mastery by taking 
the difficulty of the items into account. Levels are based on local 
decision-making. Your description may include: 

• the actual process used to determine mastery 
levels/identify facilitator of process with qualifications. 

• how the group worked toward consensus or agreement on 
mastery levels, that this was not just an arbitrary decision. 



□ Yes . . . □ No. . . 2. Students were classified at varying mastery levels and the scores 

obtained by students in the various classifications were used to determine 
the score needed for mastery. These classifications were decisions made 
by the local district. Your description may include: 

• how actual student work was used in the classification process. 

• the people/number/qualifications of those who made these 
decisions about mastery levels 

• the identify of those who facilitated this process/qualifications. 



□ Yes ... □ No... 3. Other: 
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This document is covered by a signed "Reproduction Release 
(Blanket)" form (on file within the ERIC system), encompassing all 
or classes of documents from its source organization and, therefore, 
does not require a "Specific Document" Release form. 
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This document is Federally-funded, or carries its own permission to 
reproduce, or is otherwise in the public domain and, therefore, may 
be reproduced by ERIC without a signed Reproduction Release form 
(either "Specific Document" or "Blanket"). 
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